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ABSTRACT 

This  paper  addresses  the  design  of  opportunistic  spectrum  ac¬ 
cess  (OSA)  where  secondary  users  are  allowed  to  sense  and 
access  multiple  channels  in  the  spectrum  without  causing  un¬ 
acceptable  interference  to  primary  users.  Integrated  in  OSA 
design  are  a  spectrum  sensor  at  the  physical  (PHY)  layer  and 
a  sensing  and  an  access  strategy  at  the  MAC  layer.  Within  the 
framework  of  partially  observable  Markov  decision  process, 
we  develop  a  separation  principle  for  the  joint  OSA  design, 
leading  to  an  explicit  optimal  design  of  the  spectrum  sen¬ 
sor  and  a  closed-form  optimal  access  strategy  when  spectrum 
sensor  and  access  strategy  are  designed  independently  across 
channels.  We  also  propose  two  heuristic  approaches  that  ex¬ 
ploit  the  correlation  among  channel  occupancies,  one  at  the 
PHY  layer  and  the  other  at  the  MAC  layer.  Simulation  re¬ 
sults  indicate  that  the  exploitation  of  channel  correlation  at 
the  PHY  layer  is  more  effective  than  that  at  the  MAC  layer, 
and  that  the  detection  capability  of  the  spectrum  sensor  can 
be  improved  by  exploiting  MAC  layer  information. 

1.  INTRODUCTION 

Opportunistic  spectrum  access  (OSA)  exploits  temporal  and 
spatial  spectrum  opportunities  resulting  from  the  bursty  traf¬ 
fic  of  primary  users  and  guard  bands  in  space.  It  has  captured 
increasing  attention  recently  due  to  its  potential  in  improv¬ 
ing  spectrum  efficiency  [1],  As  shown  in  [2, 3],  basic  design 
components  of  OSA  include  (i)  a  spectrum  sensor  at  the  phys¬ 
ical  (PHY)  layer,  which  identifies  spectrum  opportunities;  (ii) 
a  sensing  strategy  at  the  MAC  layer,  which  specifies  which 
channels  in  the  spectrum  to  sense;  and  (iii)  an  access  strategy, 
also  at  the  MAC  layer,  which  determines  whether  to  access 
based  on  potentially  erroneous  sensing  outcomes.  The  design 
objective  is  to  maximize  the  throughput  of  secondary  users 
under  the  constraint  that  the  probability  of  colliding  with  pri¬ 
mary  users  is  capped  below  a  certain  threshold. 

Most  existing  work  on  OSA  focuses  on  the  cognitive  MAC 
design  under  the  ideal  assumption  that  the  spectrum  sensor  is 
perfect  [4, 5].  Recently,  within  the  framework  of  partially  ob¬ 
servable  Markov  decision  process  (POMDP),  the  design  of 
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the  PHY  layer  spectrum  sensor  has  been  integrated  into  the 
MAC  design  for  optimal  OSA  in  the  presence  of  sensing  er¬ 
rors  [2,3].  In  [2],  a  separation  principle  is  established  for  OSA 
with  single-channel  sensing,  which  leads  to  an  explicit  opti¬ 
mal  design  of  the  spectrum  sensor  and  a  closed-form  optimal 
access  strategy. 

This  paper  extends  the  results  in  [2]  to  the  scenario  where 
secondary  users  can  sense  and  access  multiple  channels  si¬ 
multaneously.  We  show  that  when  the  spectrum  sensor  and 
the  access  strategy  are  designed  independently  across  chan¬ 
nels,  the  separation  principle  developed  in  [2]  still  holds  for 
OSA  with  multi-channel  sensing.  We,  however,  note  that  such 
independent  design  is  suboptimal  since  it  ignores  the  cor¬ 
relation  among  channel  occupancies.  We  thus  propose  two 
heuristic  approaches  to  exploit  channel  correlation,  one  at  the 
PHY  layer  and  the  other  at  the  MAC  layer.  Simulation  re¬ 
sults  indicate  that  the  exploitation  of  channel  correlation  at 
the  PHY  layer  is  more  effective  than  that  at  the  MAC  layer. 
We  also  find  that  the  performance  of  the  PHY  layer  spectrum 
sensor  improves  over  time  by  incorporating  the  MAC  layer 
sensing  and  access  decisions.  These  observations  illustrate 
the  two-way  interaction  between  the  PHY  and  the  MAC  lay¬ 
ers:  the  necessity  of  incorporating  the  sensor  operating  char¬ 
acteristics  into  the  MAC  design  and  the  benefit  of  exploiting 
the  MAC  layer  information  in  the  PHY  layer  design. 

2.  PROBLEM  STATEMENT  AND  FORMULATION 
2.1.  Network  Model 

Consider  a  spectrum  of  N  orthogonal  channels  licensed  to  a 
slotted  primary  network.  Let  Sn(t)  £  {0  (busy),  1  (idle)}  de¬ 
note  the  occupancy  of  channel  n  in  slot  t.  We  assume  that 

the  spectrum  occupancy  S(i)  =  [S'i(f), . . . ,  jSjv(t)]  follows  a 
discrete-time  homogeneous  Markov  process  with  finite  state 
space  §  =  {0, 1}^.  The  transition  probabilities  are  denoted 

as  {Ps, s'}s,s'€S,  where  Ps  s>  =  Pr{S(t  +  1)  =  s'  |  S(t)  =  s} 

is  the  probability  that  the  spectrum  occupancy  state  transits 
from  s  G  S  to  s'  G  S.  We  assume  that  the  transition  probabil¬ 
ities  are  known  and  remain  unchanged  in  T  slots1. 

1  The  robustness  of  the  optimal  OSA  design  to  inaccurate  transition  prob¬ 
abilities  has  been  demonstrated  in  [2].  When  the  transition  probabilities  are 
unknown,  formulations  and  algorithms  for  POMDP  with  an  unknown  model 
exist  in  the  literature  [6]  and  can  be  applied  to  this  problem. 
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We  consider  a  secondary  ad  hoc  network  whose  users  in¬ 
dependently  search  for  and  exploit  instantaneous  spectrum 
opportunities  in  these  N  channels2.  Specifically,  at  the  begin¬ 
ning  of  slot  t,  a  secondary  user  with  data  to  transmit  chooses  a 
set  A(t)  C  {1, . . . ,  N}  of  channels  to  sense,  where  |„4(t)|  = 
L  and  1  <  L  <  N.  Based  on  the  imperfect  sensing  out¬ 
comes  ®A{t)  =  {<dn{t)}n£A(t)  G  {0  (busy),  1  (idle)}L,  the 
secondary  user  decides  whether  to  access  each  sensed  chan¬ 
nel:  &A(t)  =  {^n(i)}neX(t)  €  (0  (no  access),  1  (access)}L. 
A  collision  with  primary  users  happens  when  the  secondary 
user  accesses  a  busy  channel.  Collisions  among  secondary 
users  are  resolved  via  carrier  sensing,  i.e.,  the  secondary  user 
with  the  smallest  backoff  time  transmits.  At  the  end  of  this 
slot,  the  receiver  acknowledges  every  successful  transmission. 
We  let  KA(t)  =  {Kn(t)}neA[t)  G  {0  (no  ACK),  1  (ACK)}L. 
We  assume  that  the  acknowledgement  is  error-free.  That  is, 
acknowledgement  Kn(t)  =  1  is  received  if  and  only  if  the 
secondary  user  accesses  an  idle  channel.  Our  goal  is  to  design 
an  OSA  strategy  for  the  secondary  user,  which  sequentially 
specifies  which  channels  in  the  spectrum  to  sense,  which  spec¬ 
trum  sensor  to  use,  and  which  sensed  channels  to  access. 

2.2.  Basic  Components  of  OSA 

Integrated  in  the  design  of  OSA  are  three  basic  components: 
a  spectrum  sensor,  a  sensing  strategy,  and  an  access  strategy. 
Spectrum  Sensor  Suppose  that  a  set  Ait)  of  channels  are  cho¬ 
sen  at  the  beginning  of  slot  t,  where  |A(f)|  =  L  >  1.  The 
spectrum  sensor  detects  the  occupancies  of  chosen  channels 
by  performing  a  2L-ary  hypothesis  test: 

'Ho  :  SA(t)  =  [1, . . . ,  1],  ...  SA{i)  =*  [0, . . . ,  0], 

where  S A(t)  =  {Sn(t)}neA(t)  *=  {0, 1}L  denotes  the  occu¬ 
pancies  of  the  chosen  channels  A(t)  in  the  current  slot.  The 
probabilities  of  these  hypotheses  are  learned  from  the  entire 
decision  and  observation  history  of  the  secondary  user. 

Sensing  errors  occur  if  the  spectrum  sensor  mistakes  one 
hypothesis  for  another.  Since  there  are  a  total  of  2L  hypothe¬ 
ses,  the  performance  of  the  spectrum  sensor  can  be  specified 
by  a  set  of2Lx  (2L  — 1)  error  probabilities.  In  the  presence  of 
noise  and  fading,  perfect  sensing  cannot  be  attained.  Hence, 
the  optimal  design  of  the  spectrum  sensor  should  achieve  a 
tradeoff  among  these  2L  x  ( 2L  —  1)  error  probabilities. 
Sensing  and  Access  Strategies  A  sensing  strategy  specifies, 
in  each  slot  t,  a  set  A(t)  of  channels  to  be  sensed,  where 
A(t)  C  {l,...,iV}  and  |A(i)|  =  L.  An  access  strategy 
decides  a  set  of  transmission  probabilities  {fn(SA,  f)},  where 

fn(0A,t)=Pr{$n{t)  =  =  0.4} 

is  the  probability  of  accessing  sensed  channel  n  G  AU)  when 
the  sensing  outcomes  are  given  by  0A(t)  =  {0, 1}L  in  slot  t,. 

2  The  design  of  cooperative  OSA  strategies  for  secondary  users  can  be 
formulated  by  a  decentralized  POMDP  problem  [7],  We  point  out  that 
while  cooperation  among  secondary  users  might  improve  the  overall  network 
throughput,  such  cooperative  design  is  much  more  complex. 


2.3.  POMDP  Formulation 

The  joint  design  of  OSA  with  multi-channel  sensing  can  be 
formulated  as  a  POMDP  problem. 

Reward  Assume  that  the  expected  number  of  bits  that  can  be 
delivered  by  the  secondary  user  in  a  slot  is  proportional  to  the 
channel  bandwidth.  Given  sensing  action  A{t),  we  define  the 
immediate  reward  R^t)l  as 

*£$)=  E  Kn(t)Bn.  (1) 

n£A.(t) 

Hence,  the  total  expected  reward  represents  overall  through¬ 
put,  the  total  expected  number  of  bits  that  can  be  delivered  by 
the  secondary  user  in  T  slots. 

Observation  As  detailed  in  [9],  the  secondary  user  and  its 
desired  receiver  must  have  the  same  history  of  observations  so 
that  they  make  the  same  channel  selection  decisions  without 
exchanging  extra  control  message.  Since  sensing  errors  may 
cause  different  sensing  outcomes  at  the  transmitter  and  the 
receiver,  the  acknowledgement  K A(t)  should  be  used  as  the 
common  observation  in  each  slot. 

Belief  Vector  Within  the  framework  of  POMDP,  the  sec¬ 
ondary  user’s  knowledge  of  the  spectrum  occupancy  based 
on  its  decision  and  observation  history  can  be  encoded  in  a 

belief  vector  A(t)  =  {As(t)}sgs  [8],  where  A s(t)  is  the  con¬ 
ditional  probability  (given  the  entire  observation  history)  that 
the  spectrum  occupancy  is  in  state  s  G  §  at  the  beginning 
of  slot  t  prior  to  the  state  transition.  We  point  out  here  that 
based  on  the  belief  vector  A(t)  at  the  beginning  of  slot  t,  the 
secondary  user  can  calculate  the  distribution  of  the  current 
spectrum  occupancy  S  (t)  as 

Pr{S(f)  =  s}  =  Y,  As'  (t)iV, s,  Vs  G  S.  (2) 

s'  GS 

The  probabilities  of  the  hypothesis  used  in  the  design  of  the 
spectrum  sensor  can  thus  be  determined  from  (2). 

Objective  The  design  objective  is  to  maximize  the  total  ex¬ 
pected  reward  in  T  slots  under  the  constraint  that  the  proba¬ 
bility  Pn(t)  of  collision  perceived  by  the  primary  network  in 
any  channel  n  and  any  slot  t  is  capped  below  a  threshold  (: 

max  E  [^2  (t)  I A  ( 1 )  ] 

TAi  (3) 

s.t.  Pn(t)  =  Pr{$„(f)  =  l|Sn(t)  =  0}<  C,  Vt, n, 

where  A(l)  is  the  initial  belief  vector,  which  represents  the 
information  on  the  initial  spectrum  occupancy.  Note  that  when 
Pr{i5n(i)  =  0}  =  0,  no  collision  will  occur  and  the  optimal 
access  decision  is  straightforward:  $„(f)  =  1. 

3.  SEPARATION  PRINCIPLE 

The  constrained  POMDP  given  in  (3)  appears  to  be  intractable 
due  to  the  high-dimensional  and  uncountable  action  space.  In 
this  section,  we  show  that  under  certain  conditions,  there  ex¬ 
ists  a  separation  principle  for  the  optimal  joint  design  of  OSA, 
which  enables  us  to  obtain  an  explicit  optimal  design  of  the 
spectrum  sensor  and  a  closed-form  optimal  access  strategy. 


Theorem  1:  When  the  spectrum  sensor  and  the  access 
strategy  are  designed  independently  across  channels,  the  op¬ 
timal  joint  design  of  OS  A  with  multi-channel  sensing  can  be 
carried  out  in  two  steps: 

1.  Choose  the  spectrum  sensor  and  the  access  strategy  to 
maximize  the  expected  immediate  reward  subject  to  the 
collision  constraint. 

2.  Choose  the  sensing  strategy  to  maximize  the  expected 
total  reward. 

In  this  case,  the  optimal  spectrum  sensor  is  given  by  the  op¬ 
timal  Neyman-Pearson  (NP)  detector  with  probability  of  miss 
detection  equal  to  f  which  detects  the  channel  occupancy  by 
using  only  the  measurements  from  this  channel,  and  the  opti¬ 
mal  access  decision  is  to  trust  the  sensing  outcome  from  this 
channel.  The  optimal  sensing  strategy  can  be  obtained  by 
solving  an  unconstrained  POMDP. 

Proof:  See  [9].  □□□ 

We  emphasize  that  the  extension  of  the  separation  prin¬ 
ciple  developed  in  [2]  to  the  multi-channel  sensing  scenar¬ 
ios  is  based  on  the  assumption  that  the  spectrum  sensor  and 
the  access  strategy  are  designed  independently  across  chan¬ 
nels.  Specifically,  we  assume  that  the  occupancy  of  a  channel 
is  detected  without  taking  into  account  the  measurements  of 
other  channels  and  the  access  decision  on  a  channel  is  made 
independently  of  the  sensing  outcomes  from  other  channels. 
Intuitively,  under  the  above  assumption,  the  design  of  the 
spectrum  sensor  and  the  access  strategy  for  the  multi-channel 
sensing  case  can  be  treated  as  L  independent  design  prob¬ 
lems,  one  for  each  chosen  channel.  Hence,  the  optimal  design 
for  the  single-channel  L  =  1  sensing  case  can  be  extended  to 
L  >  1. 

Theorem  1  provides  sufficient  conditions  under  which  the 
design  given  by  the  separation  principle  (referred  to  as  the  SP 
approach  for  simplicity)  is  optimal.  In  Proposition  1,  we  show 
that  the  SP  approach  is  locally  optimal  ( i.e .,  maximizes  the 
expected  immediate  reward)  under  certain  relaxed  conditions. 

Proposition  1:  Suppose  that  the  spectrum  sensor  is  de¬ 
signed  independently  across  channels  while  the  access  strat¬ 
egy  jointly  exploits  the  sensing  outcomes  from  all  channels. 
The  SP  approach  is  locally  optimal  when  channel  occupan¬ 
cies  are  independent. 

Proof:  See  [9].  □□□ 

4.  HEURISTIC  APPROACHES 

While  simplifying  the  joint  design  of  OS  A  with  multi-channel 
sensing,  the  condition  that  the  spectrum  sensor  and  the  ac¬ 
cess  strategy  are  designed  independently  across  channels  re¬ 
sult  in  throughput  degradation  since  the  correlation  among 
channel  occupancies  is  ignored.  We  propose  two  heuristic 
approaches  to  exploit  the  channel  correlation:  the  PHY  layer 
and  the  MAC  layer  approaches. 

4.1.  The  PHY  Layer  Approach 

When  the  spectrum  occupancy  states  are  correlated  across 
channels,  we  have  correlated  channel  measurements  at  the 


PHY  layer.  Hence,  the  channel  correlation  can  be  exploited 
by  using  the  measurements  of  all  chosen  channels  in  occu¬ 
pancy  detection.  With  this  in  mind,  we  propose  a  heuristic 
design  of  the  spectrum  sensor:  it  adopts  the  optimal  NP  de¬ 
tector  with  probability  of  miss  detection  equal  to  <j,  which 
is  designed  to  uses  all  channel  measurements.  We  note  that 
the  structure  of  the  optimal  NP  detector  adopted  by  this  sen¬ 
sor  relies  on  the  joint  distribution  of  the  spectrum  occupancy 
states,  which  is  given  by  the  belief  vector  (see  Section  5  for 
an  example).  That  is,  this  heuristic  sensor  design  is  affected 
by  the  observation  and  decision  history  of  the  secondary  user 
and  thus  improves  over  time  due  to  accumulated  observations 
(see  Figure  1). 

Based  on  the  sensing  outcomes  given  by  the  above  spec¬ 
trum  sensor,  the  secondary  user  can  adopt  the  access  strategy 
of  the  SP  approach:  to  access  if  and  only  if  the  channel  is 
sensed  as  idle.  We  refer  this  approach  as  the  PHY  layer  ap¬ 
proach.  Proposition  2  provides  a  sufficient  condition  under 
which  this  PHY  layer  approach  is  locally  optimal. 

Proposition  2:  Suppose  that  the  access  strategy  is  de¬ 
signed  independently  across  channels  while  the  spectrum  sen¬ 
sor  jointly  exploits  the  measurements  taken  from  all  chosen 
channels.  The  PHY  layer  approach  is  locally  optimal.  When 
channel  occupancies  are  independent,  the  PHY  layer  approach 
reduces  to  the  SP  approach. 

Proof:  See  [9].  □□□ 

4.2.  The  MAC  Layer  Approach 

When  channel  occupancies  are  correlated,  so  are  the  sensing 
outcomes  given  by  the  spectrum  sensor.  Hence,  the  chan¬ 
nel  correlation  can  also  be  exploited  at  the  MAC  layer  by 
making  access  decisions  jointly  across  channels.  A  heuris¬ 
tic  MAC  layer  approach  is  to  adopt  the  SP  sensor,  which  de¬ 
tects  the  channel  occupancy  by  using  only  the  measurements 
of  this  channel,  and  then  choose  the  access  decisions  that  ex¬ 
ploit  sensing  outcomes  from  all  chosen  channels  to  maximize 
the  expected  immediate  reward.  Specifically,  for  any  chosen 
channels  A{t )  and  any  belief  vector  A(f)  in  slot  t,  we  choose 
the  set  of  transmission  probabilities  as  follows 

{fn(0A,t)}  =  arg  max  |A(f)] 

=  arg  max  Y'  BnPr{S„(t)  =  1} 
/n(0^)e[o,i]n“() 

x  J2  hsA\sn(sA\l)l&AlSA(eA\sA)  fn{0A), 

0A<SA 

S.t.  Pn{f)  =  ^  hSA\Sn(SA  |  0) 

0A,SA 

xl&A\sA(eA\sA)fn(0A)<C,  VneA(I), 

where  hSA\Sn(sA\i)  =  Pr{SA(t)  =  sA\Sn(t)  =*},*  = 
0, 1,  is  the  conditional  distribution  of  the  channel  occupancy 
states  SA(t),  which  can  be  obtained  from  the  belief  vector  via 
(2),  and  1&a\Sa{8a  |  s^)  =  Pr{0^(f)  =  9A  |  S A(t)  =  s^} 


is  the  sensing  error  probability  determined  by  the  operating 
characteristics  of  the  SP  sensor. 

We  can  obtain  the  above  access  strategy  via  linear  pro¬ 
gramming.  Proposition  3  shows  that  this  MAC  layer  approach 
is  equivalent  to  the  SP  approach  when  channel  occupancies 
are  independent.  This  agrees  with  our  intuition  that  when 
channels  are  independent,  so  are  the  sensing  outcomes  from 
the  chosen  channels.  Hence,  independent  access  decision¬ 
making  performs  as  well  as  the  joint  one  in  terms  of  immedi¬ 
ate  reward. 

Proposition  3:  Suppose  that  the  spectrum  sensor  is  de¬ 
signed  independently  across  channels  while  the  access  strat¬ 
egy  jointly  exploits  the  sensing  outcomes  from  all  chosen  chan¬ 
nels.  When  channel  occupancies  are  independent,  the  MAC 
layer  approach  reduces  to  the  SP  approach  and  hence  is  lo¬ 
cally  optimal. 

Proof:  See  [9],  □□□ 


5.  SIMULATION  EXAMPLES 


In  this  section,  we  compare  the  performance  of  the  SP,  the 
PHY  layer,  and  the  MAC  layer  approaches.  Since  these  three 
approaches  differ  in  the  spectrum  sensor  and  the  access  strat¬ 
egy,  we  can  employ  any  sensing  strategy  to  compare  their  per¬ 
formance.  For  simplicity,  we  adopt  a  myopic  sensing  strategy 
that  chooses,  in  each  slot,  the  set  A  of  channels  that  maxi¬ 
mizes  the  expected  immediate  reward  under  perfect  sensing, 

A  =  arg  nmx  ^  B„  Pr{S„(t)  =  1}.  (4) 

n£A. 

We  model  both  the  background  noise  and  the  primary  sig¬ 
nal  as  white  Gaussian  processes.  Let  <Tq  and  erf  denote  the 
noise  power  and  the  primary  signal  power,  respectively.  At 
the  beginning  of  each  slot,  the  spectrum  sensor  takes  M  inde¬ 
pendent  measurements  Yn=[Ynp , . . . ,  Y^m]  from  each  cho¬ 
sen  channel  n  £  A. 

For  each  chosen  channel  n  £  A.  the  SP  sensor  uses  the 
corresponding  measurements  Yn  and  performs  the  following 
hypothesis  test: 

'Ho{Sn  =  1)  :  Yn  ~  M{Om ,  OqIm), 

Hi(Sn=0):  Yn~Af(OM,(a?  +  o-g)lM), 


where  <t2Im)  denotes  an  M-dimensional  Gaussian 

distribution  with  identical  mean  0  and  variance  a2  in  each  di¬ 
mension.  It  can  be  readily  shown  that  the  optimal  NP  detector 
is  given  by  an  energy  detector  [10,  Sec.  2.6.2]: 

M 

WYn\\2  =  J2Yli^nlVn.  (6) 

i=  1 

The  probabilities  of  false  alarm  en  and  miss  detection  6n  of 
the  energy  detector  can  be  calculated  by  [10,  Sec.  2.6.2]: 


5n  =  7 


t)n 


M 


en  =  1  -  7 


M  T)n 
2  ’  2cr§ 


where  7(m,  a)  =  j=^y  JQa  xe  4  dt  is  the  incomplete  gamma 
function.  The  optimal  decision  threshold  rf*n  of  the  energy  de¬ 
tector  is  chosen  so  that  the  probability  of  miss  detection  is 
fixed  at  5n  =  (. 


On  the  other  hand,  the  sensor  of  the  PHY  layer  approach 
uses  all  channel  measurements  { Y„  }nc_4  and  performs  a  com¬ 
posite  hypothesis  test  for  each  chosen  channel  n  £  A: 

/Ho{Sn(t)  =  1)  :  Yn  ~  A/XOMjOoIm), 

Yra  ~  Af(0 M,  ( (To  +  l[Sm=0]Ci)lM),  Vm  e  -4\{n} 

Hi (Sn(t)  =  0)  :  Y „  ~  W(0m,  (cro  +  <7i)Im), 

Y m  ~  M(0m,  {o o  +  l[5m=0]CTi)IM),  Vm  £  A\{n}, 


The  distribution  of  the  channel  occupancy  states  S ^(t)  under 
each  hypothesis  is  given  by  hsA\sn(sA  |  *)•  The  optimal  NP 
detector  for  (7)  is  a  likelihood  ratio  test  [10,  Sec.  2.5]: 

^i5B(sx|0)nme^MYmM  ^ni  _ 
E^^is„(sx|i)nmexP*»(Ymkm)  <Wo  r"’ 
where  p.n  (Y„|sn)  is  the  probability  density  function  of  inde¬ 
pendent  Gaussian  channel  measurements  Yn: 


2t<Tn  +  1T. 


=o]°i> 


Pn(Y„|s„)  —  J^[  t  Wf( 

t=l  +  l[s„=0]Crr) 

Note  that  when  channel  occupancies  are  independent,  the  sen¬ 
sor  employed  by  the  PHY  layer  approach  is  equivalent  to  the 
SP  sensor  since  hSA\Sri  (s^  |  0)  =  hSA\Sn(slA  \  1)  for  any 
and  such  that  sm  =  s'm  when  m  n.  The  error  proba¬ 
bilities  of  this  sensor  can  be  evaluated  via  simulation.  In  each 
slot,  the  optimal  detection  threshold  r*  is  chosen  according 
to  the  belief  vector  so  that  the  resulting  probability  of  miss 
detection  is  fixed  at  (. 

As  proven  in  Propositions  2  and  3,  the  PHY  layer  and 
the  MAC  layer  approaches  are  equivalent  to  the  SP  approach 
when  channel  occupancies  are  independent.  We  thus  com¬ 
pare  below  the  performance  of  these  three  approaches  in  cor¬ 
related  channels.  Specifically,  we  consider  N  =  4  correlated 
channels,  each  with  bandwidth  Bn  =  1.  The  transition  prob¬ 
abilities  of  the  spectrum  occupancy  are  given  by 

F’[oooo],[oni]  =  0.6,  -P[oooo],[oooo]  =  0.4, 

-P[0111],[0000]  =  -P[1011],[0000]  =  -P[1101],[0000]  =  -P[1110],[0000]  =  0.2, 


-P[oni],[ioii]  =  -P[10ii],[noi]  =  -Fjnoijjino]  =  -Fjmojjoiii]  =  0.8. 
For  simplicity,  we  assume  that  there  is  only  one  secondary 
user  seeking  instantaneous  spectrum  availability.  The  initial 
belief  vector  is  set  to  the  stationary  distribution  of  the  under¬ 
lying  Markov  process.  The  maximum  allowable  probability 
of  collision  is  (  =  0.05.  In  each  slot,  L  =  3  channels  are  cho¬ 
sen.  The  spectrum  sensor  takes  M  =  1  measurement  of  each 
chosen  channel  and  the  noise  and  the  primary  signal  powers 
are  given  by  (Tq  =  0  dB  and  a 2  =  10  dB. 


5.1.  Comparison  of  Sensor  Performance 

In  Figure  1 ,  we  plot  the  receiver  operating  characteristic  (ROC) 
curves  (probability  of  false  alarm  vs.  probability  of  detec¬ 
tion)  of  the  SP  sensor  and  the  sensor  employed  by  the  PHY 
layer  approach.  Note  that  the  sensor  of  the  MAC  layer  ap¬ 
proach  is  the  same  as  the  SP  sensor.  We  see  that  the  sensor  of 
the  PHY  layer  approach  outperforms  the  SP  sensor.  Specif¬ 
ically,  for  a  fixed  probability  of  miss  detection,  the  probabil¬ 
ity  of  false  alarm  of  the  sensor  of  the  PHY  layer  approach 
is  much  smaller  than  that  of  the  SP  sensor.  This  is  because 


Fig.  1.  Comparison  of  ROC  curves. 


the  PHY  layer  approach  exploits  the  correlation  among  chan¬ 
nel  measurements  in  occupancy  detection  while  the  SP  sensor 
does  not.  We  also  observe  that  the  ROC  curve  of  the  sensor 
employed  by  the  PHY  layer  approach  improves  in  each  slot 
while  that  of  the  SP  sensor  remains  the  same.  This  obser¬ 
vation  can  be  explained  by  comparing  the  optimal  detectors 
(6)  and  (8).  Clearly,  the  energy  detector  (6)  used  by  the  SP 
approach  is  static  and  so  is  its  performance.  However,  as 
seen  from  (8),  the  decision  variable  of  the  sensor  employed 
by  the  PHY  layer  approach  depends  on  the  conditional  distri¬ 
bution  hsA\sn(sA  I  i)  °f  the  channel  occupancies.  Hence,  its 
performance  varies  over  time  according  to  the  belief  vector. 
As  time  t  increases,  the  belief  vector  and  hence  the  sensor 
performance  improves  due  to  the  accumulated  observations. 
Figure  1  demonstrates  that  the  performance  of  the  PHY  layer 
sensor  can  be  improved  by  incorporating  the  MAC  layer  sens¬ 
ing  and  access  decisions  encoded  in  the  belief  vector. 

5.2.  Comparison  of  Throughput  Performance 


Fig.  2.  Comparison  of  normalized  throughput. 

In  Figure  2,  we  compare  the  throughput,  measured  by  the 
total  expected  reward  per  slot,  of  these  three  approaches.  As 
expected,  the  SP  approach,  which  ignores  the  channel  cor¬ 
relation,  performs  the  worst.  By  jointly  exploiting  the  sens¬ 
ing  outcomes  in  access  decision-making,  the  MAC  layer  ap¬ 
proach  can  improve  throughput  performance.  A  much  larger 
performance  gain  is  achieved  by  the  PHY  layer  approach  which 
jointly  exploits  the  channel  measurements  in  occupancy  de¬ 
tection.  We  can  thus  see  that  the  exploitation  of  channel  cor¬ 
relation  at  the  PHY  layer  is  more  effective  than  that  at  the 
MAC  layer.  In  other  words,  independent  channel  occupancy 
detection  at  the  PHY  layer  hurts  throughput  more  than  inde¬ 


pendent  access  decision-making  at  MAC  layer.  This  agrees 
with  our  intuition  because  the  spectrum  sensor  of  the  MAC 
layer  approach  makes  a  hard-decision  on  whether  the  channel 
is  idle  or  not.  The  resulting  sensing  outcomes  are  thus  less 
informative  than  the  original  channel  measurements,  leading 
to  throughput  degradation. 

6.  CONCLUSION 

In  this  paper,  we  provided  sufficient  conditions  for  the  exis¬ 
tence  of  separation  principle  for  optimal  design  of  OSA  with 
multi-channel  sensing.  We  also  proposed  two  heuristic  ap¬ 
proaches  that  exploit  the  correlation  among  channel  occupan¬ 
cies.  Simulation  examples  demonstrated  the  performance  im¬ 
provement  of  the  PHY  layer  spectrum  sensor  over  time  re¬ 
sulting  from  the  incorporation  of  the  MAC  layer  sensing  and 
access  decisions.  We  also  found  that  the  exploitation  of  chan¬ 
nel  correlation  at  the  PHY  layer  is  more  effective  than  that  at 
the  MAC  layer  in  improving  throughput. 
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Spectrum  Scarcity  vs.  Spectrum  Opportunity 

Pervasive  Spectrum  Opportunities 
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Spectrum  usage  of  an  active  FTP  session  in  a  WLAN  (ACSP  at  Cornell). 

►  Almost  all  usable  radio  frequencies  have  already  been  licensed. 

►  At  any  given  time  and  location,  a  large  portion  of  licensed  spectrum  lies  unused. 

□  Over  62%  white  space  exists  in  the  spectrum  under  3GHz. 

□  About  75%  idle  time  during  an  active  FTP  session  in  WLAN. 

□  Up  to  90%  idle  time  during  voice-over-IP  applications  such  as  Skype. 


Overly  Crowded  Spectrum 
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Opportunistic  Spectrum  Access  (OSA) 


Opportunities 


Basic  idea: 


Allow  secondary  users  to  ex¬ 
ploit  spectrum  opportunities. 


Design  objective: 

Maximize  secondary  users’ 
throughput  while  limiting 
Secondary  user  t|iejr  interference  to  primary 

users  (licensees). 


Primary  user 


Three  basic  design  components: 

►  Spectrum  sensor:  opportunity  identification  (PHY) 

►  Sensing  policy:  where  in  the  spectrum  to  sense  (MAC) 

►  Access  policy:  whether  to  tx  given  sensing  errors  may  occur  (MAC) 

A  decision-theoretic  framework  for  joint  PHY-MAC  design. 
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Main  Results 


►  A  decision-theoretic  framework  based  on  partially  observable  Markov  decision 
process  (POMDP) 

□  jointly  optimizes  the  three  basic  design  components 

□  captures  fundamental  design  tradeoffs 
Spectrum  sensor:  false  alarm  vs.  miss  detection 
Access  Policy:  overlooked  opportunity  vs.  collision 
Sensing  Policy:  gaining  access  vs.  gaining  information 

►  Structural  policies  for  the  joint  design 

□  Separation  principle  for  single-channel  sensing  and  its  extension  to 
multi-channel  sensing 

□  Explicit  optimal  sensor  design  and  closed-form  optimal  access  policy. 

□  Sensing  design  reduced  from  a  constrained  POMDP  to  an  unconstrained  one. 

►  Quantitative  characterization  of  the  interaction  between  PHY  and  MAC 

□  Impact  of  the  operating  characteristics  of  spectrum  sensor  on  MAC 

□  Exploiting  MAC  information  at  PHY  for  improved  sensor  performance. 
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Network  Model 


SN(  1)  =  1  Sn(2)  =  0  Sn(3)  =0  Sn{T)  =  0 

►  A  spectrum  of  TV  channels,  each  with  bandwidth  Bn. 

►  A  slotted  primary  network 

□  Markovian  spectrum  usage  with  2N  states: 

s (t)  =  [Si(t), . . . ,  SN(t)\  e  {0  (busy),  1  (idle)}^. 

□  Known  transition  probabilities. 

►  An  ad  hoc  secondary  network  without  dedicated  control  channel 

□  Independent  users,  each  can  sense  and  access  L  channels  in  each  slot. 

□  User  obtains  a  reward  R(t)  =  Bn  for  each  successful  access  of  idle  channels. 

□  User  collides  with  primary  users  if  access  a  busy  channel. 
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Basic  Components  and  Design  Tradeoffs 


Spectrum  Sensor:  false  alarm  vs.  miss  detection 

►  Binary  hypotheses  test  (for  L  =  l): 

Ho :  channel  is  idle  vs.  Hi  :  channel  is  busy 

►  Two  Types  of  sensing  errors: 

□  false  alarm  (e):  Ho^Hi  (overlook) 

□  miss  detection  (s)\  Hi—>Ho  (misidentification) 


►  Which  point  (e,6)  on  the  ROC  curve  should  the  sensor  operate?  (decision  rule) 
Access  Policy:  overlooked  opportunity  vs.  collision 

►  Consequences  of  trusting  sensing  outcome: 

□  false  alarm  (idle  sensed  as  busy)  =>*  overlooked  opportunity 

□  miss  detection  (busy  sensed  as  idle)  =^>  collision 

►  When  and  how  much  to  trust  the  sensor? 


For  L  =  1,  tx  prob.  = 


Po  if  idle 
Pi  if  busy 


Po  <1  :  conservative 


<£=>  sensor  {e,  6} 


Pi  >  0  :  aggressive 

Joint  design  of  access  policy  at  MAC  and  spectrum  sensor  at  PHY 
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Basic  Components  and  Design  Tradeoffs 


Sensing  Policy:  gaining  access  vs.  gaining  information 


Reward  #(1) 
Action  (  -4.(1)  | 

m 

A(2)  ^ 

R{t- 1);'  r 

*  ^ 

•  •  •  |  A(t  -  1|  A 

:t)  R(t  + 1)  •  •  • 

t)  ^  A(t  +  1)^  ^ 

. . 

'  t  +  1  j 

Observ. 

K(2) 

K(J,  -  Ij.-'K 

-^A  (t  +  1) 

►  Each  observation  K (t)  e  (o(unsuccessful),  l(successful  access)}L  provides  partial 
information  on  the  spectrum  usage  state. 

►  Sensing  action  A(t)  should  be  based  on  the  conditional  distribution  A (t)  that 
exploits  the  entire  decision  and  observation  history. 

►  A(t)  results  in  immediate  reward  R(t)  and  observation  K (t)  that  affects  future 
reward. 

►  Optimal  A{t)  achieves  the  best  tradeoff  between  gaining  immediate  reward  and 
gaining  spectrum  information. 
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A  Constrained  POMDP  Formulation 


A(t)  A(t+1) 


-  slot  t  - 

Decision 

Making 

Data  Transmission 

(Obtain  reward  if  channel  idle/  Incur  collision  if  channel  busy) 

Ack.  K (t) 

(tx  succeeds  or  not) 

Sensing  decision  A(t) 

Sensor  design  A (t) 

Sensing  outcome  ©(/) 

Access  decision  <&(/) 

►  Belief  vector  A (t)  =  {As(t)}se{0)ip,  where  A s(t)  is  the  conditional  probability  that 
the  spectrum  is  in  state  s:  {A(t),A{t),  K(t)}  — >  A(t  +  1) 

►  Sensing  policy  ns 

□  deterministic:  A (t)  — >  a  set  A(t)  of  L  channels  to  be  sensed  in  slot  t. 

□  randomized:  A (£)  — >  PMF  of  A{t). 

►  Spectrum  sensor  ns\  A (t)  -►  a  decision  rule  A (t)  used  for  occupancy  detection: 

{A(t),  channel  measurements}  — >  ©(£)  =  e  (O(busy),  l(idle)}L. 

►  Access  policy  nc 

□  deterministic:  {A (£),©(£)}  ->  &(t)  e  {0(no  access),  l(access)}L. 

□  randomized:  {A (£),©(£)}—►  tx  probabilities. 

►  Objective: 

{7r|, 7T*, 7r*}  =  arg  max  R{t)\  s.t.  collision  prob.  Pn(t)  <  f,  Vn  G  A(t)  (*) 

KS^s,  7TC  J 
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The  Separation  Principle  for  Single-Channel  Sensing 


Theorem:  ns  and  nc  can  be  decoupled 


□  Choose  and  nc  to  max.  immediate  reward  R(t)  and  ensure  constraint  pn(t)  =  C- 
==>•  A  static  optimization  problem. 

Explicit  optimal  design  of  spectrum  sensor: 

optimal  Neyman-Pearson  (NP)  detector  with  prob.  of  missing  (PM)  5  =  C- 
Closed-form  optimal  access  policy: 
trust  sensing  outcome  (tx  prob.  Po  =  0,Pi  =  l). 

i-C 


□  Choose  n s  to  maximize  total  reward  E 
An  unconstrained  POMDP. 


>  Deterministic  sensing  policy. 
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Two  Spectrum  Sensor  Structures  for  Multi-Channel  Sensing 


Goal:  dynamically  choose  decision  rules  A (t)  for  spectrum  opp.  identification. 

Joint  opportunity  identification  Independent  opportunity  identification 


Channel  Yi 

Measurements  > 

y2 

'  \ 

.  yl 

1  ' 

1 

A(t)  ^ 

Spectrum  sensor  (decision  rule  A (t)) 

Sensing 

Outcomes 

0i 

' 

©2 

' 

.  0/. 

1 

1 

►  Perform  a  2i-ary  hypothesis  test: 


SA(t)  =  [1,1, 

'Hi  ■  SA{t)  =  [0,1, • 

H2l- i:  SA(t)  =  [0,0,..., 0]. 

►  Decision  rule:  {{Yn}£=1}^{w0,..-,w2t-i} 

jointly  exploits  channel  measurements. 

►  Performance  is  specified  by  a  set 
of  2L  x  ( 2l  -  l)  error  probabilities. 


►  Performs  L  independent  hypothesis 
tests:  n0-.  s„(t)  =  i, 

Hi  :  Sn(t )  =  0,  n  G  A(t). 

►  Decision  rule  a n(t)  ■.  {Yn}  ->  {H^Hi} 

ignores  correlation  among  channel 
measurements. 

►  Performance  is  specified  by  L  pairs 
of  false  alarm  and  miss  detection  rates. 

►  Less  complex  than  joint  identification. 
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Two  Access  Policy  Structures  for  Multi-Channel  Sensing 


Goal:  dynamically  choose  access  decisions  or  transmission  probabilities. 

Joint  access  decision-making  Independent  access  decision-making 


►  Tx.  probability  /„(©)  =  Pr{<I>n  =  l|©} 

governs  access  decision  <Pn,Vn  e  A{t). 

►  Access  decision  jointly  exploits  sensing 
outcomes  from  all  sensed  channels: 

►  #  of  tx.  probabilities  to  be  designed 
=  2l  (possible  sensing  outcomes) 

x  L  (chosen  channels). 


►  Tx.  probability  /„(©„)  =  Pr{4>„  =  1|0„} 

independent  of  sensing  outcomes 
from  other  channels. 

►  Access  decision  ignores  correlation 
among  sensing  outcomes. 

►  #  of  tx.  probabilities  to  be  designed 
=  2 L  (chosen  channels). 

►  Less  complex  than  joint  identification. 
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Extension  of  the  Separation  Principle 

T 

{7r|, 7t*, 7r*}  =  arg  max  R{t)\  s.t.  collision  prob.  Pn(t)  <  £,  \/n  E  A(t)  (*) 

Joint  sensor  &  joint  access  structure 

►  Provides  globally  optimal  solution. 

►  Requires  randomized  policies  for  optimality. 

►  Optimal  but  computationally  prohibitive. 

Independent  sensor  k  independent  access  structure 

►  The  separation  principle  holds. 

►  Optimal  solution  under  this  structure  (the  SP  approach): 

□  Spectrum  sensor:  optimal  NP  detector  with  PM  =  (. 


►  Caveat:  ignores  correlation  among  channel  occupancies. 
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Exploiting  Correlation:  The  PHY  Layer  Approach 


Joint  sensor  &  independent  access  structure 


□  Sensor:  optimal  NP  detector,  using  all  channel  measurements,  with  PM  =  £. 

Yi  — * 


Y 


NP  detector 
for  channel  n 


Or 


L 


□  Access  policy:  trust  the  sensing  outcome  (using  one  sensing  outcome). 


Access  decision-maker 

On  * 

for  channel  n 

►  Exploits  all  channel  measurements  {Yn}neA^  in  occupancy  detection  for  each 
chosen  channel. 

►  Uses  MAC  layer  information  at  PHY  layer:  the  a  priori  joint  distribution  of 
channel  measurements  is  obtained  from  the  belief  vector  A (t). 

►  Locally  optimal  (maximizes  instantaneous  throughput). 

►  Reduces  to  the  SP  approach  when  channels  evolve  independently. 
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Exploiting  Correlation:  The  MAC  Layer  Approach 


Independent  sensor  &  joint  access  structure 


□  Sensor:  optimal  NP  detector,  using  single  channel  measurements,  with  PM  = 

_ ^  NP  detector  _ ^ 

Yn  for  channel  n  On 


□  Access  policy:  myopic  tx  probabilities  /„(©)  (maximizes  the  instantaneous 
throughput)  obtained  via  linear  programming. 

©r 
01 

►  Exploits  all  sensing  outcomes  ©  =  {04^)  in  making  access  decision  for  each 
chosen  channel. 

►  Locally  optimal  when  channels  evolve  independently. 

►  Reduces  to  the  SP  approach  when  channels  evolve  independently. 


Access  decision-maker 
for  channel  n 


<K„  ~  /„(©) 


Probability  of  Detection  1 
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Performance  Comparison 


ROC  of  spectrum  sensor  Normalized  Throughput 


►  Performance  of  the  PHY  layer  spectrum  sensor  is  improved  by  exploiting 

the  MAC  layer  sensing  and  access  decisions  (characterized  by  the  belief  vector). 

►  Exploitation  of  channel  correlation  at  the  PHY  layer  is  more  effective  than  that 
at  the  MAC  layer. 
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Conclusions 


►  Optimal  OSA  with  multi-channel  sensing  formulated  as  a  constrained  POMDP. 

►  Separation  Principle 

□  holds  for  L  =  1;  extends  to  L  >  1  under  the  independent  sensor  k  independent 
access  structure. 

□  leads  to  explicit  optimal  sensor  design  and  closed-form  optimal  access  policy. 

□  reduces  sensing  design  from  a  constrained  POMDP  to  an  unconstrained  one. 

►  Exploiting  Channel  Correlation 

□  PHY  layer  approach:  improved  sensor  performance  by  exploiting  MAC 
information  (belief  vector). 

□  MAC  layer  approach:  infers  channel  correlation  from  sensing  outcomes. 

Limitations 

►  Known  transition  probabilities  of  the  underlying  Markov  process  (robustness  to 
model  mismatch  can  be  found  in  [1]). 

►  Interaction  among  secondary  users  not  taken  into  account  (exploited  in  [2]). 
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